
United States Patent and Trademark Office 


UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 


APPLICATION NO. 

FILING DATE 

FIRST NAMED INVENTOR 

ATTORNEY DOCKET NO. 

CONFIRMATION NO. 

09/855,090 

05/14/2001 

Reiner Kraft 

ARC920000146US1 

,8622 


23334 7590 07/12/2004 

FLEIT, KAIN, GIBBONS, GUTMAN, BONGINI 
& BIANCO P.L. 

ONE BOCA COMMERCE CENTER 

551 NORTHWEST 77TH STREET, SUITE 1 1 1 

BOCA RATON, FL 33487 


EXAMINER 


ROBINSON, GRETA LEE 


ART UNIT 


PAPER NUMBER 


2177 

DATE MAILED: 07/12/2004 


f 


Please find below and/or attached an Office communication concerning this application or proceeding. 


PTO-90C (Rev. 10/03) 


m 

Application No. 

Applicant(s) 


09/855,090 

KRAFT ET AL. 

Office Action Summarv 



Examiner 

Art Unit 



Greta L. Robinson 

2177 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 


Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )K Responsive to communication(s) filed on 1-24. 26 and 27 . 
2a)K This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) 1-24.26 and 27 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-24, 26 and 27 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)Q All b)D Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . Claims 1-24, 26 and 27 are pending in the present application. 


2. Claim 25 has been cancelled; and claims 1-5, 8, 9, 20-22 and 24 have been 
amended. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 


4. Claims 1-24, 26 and 27 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Barber et al. US Patent 5,579,471 . 

Regarding claims 1-8, Barber et al. teaches a method of providing a query, the 
method comprising the steps of: 

selecting at least one non-textual language independent visual representation of 
a search object on a display; 

dropping the at least one non-textual language independent visual representation 
of the search object on a display onto at least one non-textual language independent 
visual representation corresponding to a domain object on a display; and in response to 
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the dropping step, providing a query [note : column 1 lines 24-39; column 5 lines 12- 
42; and figure 1]. The citations have been provided below for convience: 


column 1 lines 24-39: 

A recent advance has freed the query function from language - 
based user operations. Tools are available in menu-driven 
dialog interfaces which enable a user to construct queries by 
manipulating icons in a dialog box on a display 
screen. The icons are arranged to symbolize database search 
requests . 

To date, database systems which store images have not 
provided an efficient means of searching a database by other 
than textual attributes of the images. However, this requires 
the expenditure of significant resources to annotate images with 
text. Furthermore, limiting queries to textual annotations 
forgoes the possibility of searching an image database by using 
pixel information from the actual images which are stored in the 
database. 

Much work has been expended to enable digital computers to 
recognize images by features which they contain. 

column 5 lines 12-42: 

Another such window 24 is an image characteristic window 
containing, in a first embodiment, a plurality of icons 
(hereinafter "thumbnails") 26 and 27. The user interface 
further provides through the cursor control 21 the 
ability to control the cursor 15 by movement of the mouse 16. 
The cursor 15 is used in the first embodiment to "drag and drop" 
thumbnails from the image characteristics window 24 to the image 
query construction window 23. A thumbnail which has been 
dragged and dropped in the image query construction window 23 is 
indicated by reference numeral 28. 

Once thumbnails have been dragged and dropped in the image 
query window 23, an option is selected denoting a RUN QUERY 
command represented on signal 
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path 30. In response, the identification and locations of all 
thumbnails dragged to and dropped in the image query window 23 
are provided to an image 

database management system 31 on a signal path 33. 

The image database management system 31 includes a query-by- 
image-content (QBIC) engine 32 which receives the RUN QUERY 
command and thumbnail identification and location information 
and, using the information and thumbnail definitions stored at 
34 , constructs a query which is used to search image data 
representations stored at 35. The results of the search are 
used by the QBIC engine 32 to construct a result list of images 
satisfying the query parameters. The identifications of these 
images are used to obtain the identified images from a set of 
images stored at 3 6 and the images in the result list are 
provided to the window control 22 for display in a results 
window, which is not shown. 


The limitations of claims 20-24, 26 and 27 parallel claims 1-8; therefore they are 
rejected under the same rationale. 

The limitations of claims 9-19 have been addressed above except for the 
following: "a query formulator" , "a result set manager" [note: QBIC engine 32 column 5 
lines 29-42; and data processing system 10 figure 1], 


Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 1 02(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

6. Claims 1-24 and 26-27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Bergman et al. US Patent 5,909,678 in view of Barber et al. US 
Patent 5,579,471. 

Regarding claim 1 , Bergman et al. teaches a method of providing a query [note: 
"graphical user interface for constructing queries" col. 2 lines 41-50], the method 
comprising the steps of; 

selecting at least one non-textual language independent visual representation of 
a search object on a display [note: col. 2 lines 51-64]; 

dropping the at least one ... visual representation of the search object on a 
display onto at least one ... visual representation corresponding to a domain object on a 
display [see: col. 2 lines 51-64]; and 

in response to the dropping step, providing a query [note: col. 3 lines 6-56; 
abstract; col. 10 lines 55-65]. 
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Bergman et al. teaches a computer system, method and program product for 
constructing queries by selecting iconic representations of subcomponent statements 
and dragging and dropping the icon onto a template. Bergman et al. does not specify 
that the visual representation be a non-textual language independent visual 
representation. Barber et al. teaches this feature [note: column 1 17-39; also see 
column 5 lines 1-42]. It would have been obvious to one of ordinary skill at the time of 
the invention to have combined the cited references because Bergman et al. teaches 
that constructing queries by dragging and dropping iconic representations may be 
implemented various ways [see column 12 lines 26-41], 

7. Regarding claims 2-8: 

(Claims 2-5) in response to the provided query, searching at least one collection 
of information ... the result set object indicates whether at least one match was found as 
a result of the search, note Bergman et al. col. 12 line 55 through col. 14 line 10. 

(Claims 6-8) further comprising the steps of formulating a text search query ... 
sending the text search query to at least one search engine note Bergman et al. figure 
6; col. 13 lines 7-8. 

8. The limitations of claims 20-21 have been addressed above in claims 1-2; 
therefore they are rejected under the same rationale. 
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9. The limitations of claims 22-24, 26 and 27, have been addressed above except 
for the following: "a graphical user interface" and "a visual search query application" 
[see: abstract; col. 15 lines 11-56; col. 5 lines 57-58; and col. 4 lines 10-33]. 

10. Regarding claim 9, Bergman et al. teaches a system comprising: 

a graphical user interface [col. 5 lines 57-58 "the graphical user interface of this 
invention enables the construction of content based queries"]; 

a graphical user interface manager communicatively coupled to the graphical 
user interface [figure 9; col. 5 lines 15-27, also note col. 12 lines 21-25]; 

a query formulator, communicatively coupled to the graphical user interface 
manager .. to provide a query" [note: figure 9, col. 5 lines 15-27; col. 12 lines 21-25]; 

a query initiator ... a lookup system ... a result set manager [col. 6 lines 13-47; 
col. 8 lines 29-44]. Bergman et al. does not explicitly show a query initiator; however 
Bergman et al. does permit integration of a variety of graphical user interface 
techniques in a single query interface to produce the search result [see col. 8 lines 14- 
17]. It would have been obvious to have used a query initiator in Bergman et al inorder 
to build up the query and subcomponents of the query as taught in Bergman et al.. 
Bergman et al. does not specify that the visual representation be a non-textual language 
independent visual representation. Barber et al. teaches this feature [note: column 1 17- 
39; also see column 5 lines 1-42]. It would have been obvious to one of ordinary skill at 
the time of the invention to have combined the cited references because Bergman et al. 


Application/Control Number: 09/855,090 Page 8 

Art Unit: 2177 

teaches that constructing queries by dragging and dropping iconic representations may 
be implemented various ways [see column 12 lines 26-41], 

1 1 . Regarding claims 1 0-1 9: 

wherein the graphical user interface manager structured to receive at least one 
user initiated command ... lookup system ... event stacker ... structured to correlate an 
at least one search result ... [note: col. 6 lines 40-64; col. 8 lines 34-48]. 

Response to Arguments 

12. Applicant's arguments with respect to claims 1-24, 26 and 27 have been 
considered but are moot in view of the new ground(s) of rejection. 

In the response Applicant argued that the claims have been amended to clarify 
that the search is conducted through non-textual language independent representa- 
tions, and that Bergman et al. teaches the use of text based icons and templates when 
formulating queries. Applicant is correct Bergman et al. does use text based icons; but 
he also teaches alternate implementations may be implemented through object 
definition see column 12 lines 18-40; also note column 1 1 lines 20-41; column 2 lines 
41-55. Bergman et al. has been combined with newly cited reference Barber et al. who 
specifically notes that the icons are used to free the query function from 
language-based user operations [see column 1 lines 24-25]. Note Barber et al. was 
cited on form PTO 1449 paper number three filed May 14, 2001. 
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Applicant's arguments and amendment overcome the drawing objection cited 
under 37 CFR 1 .83(a) and claim rejections cited under 35 USC 112 second paragraph. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

14. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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1 5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Greta L. Robinson whose telephone number is (703) 
308-7565. The examiner can normally be reached on Mon.-Fri. 9:30AM-6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John E. Breene can be reached on (703) 305-9790. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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Greta Robinson 
Primary Examiner 
July 2, 2004 


